S8.1 Policy sensitivity test: Scenario input conditions
Introduction
This document presents the results assessing the impact of input assumptions on policy behavior for the Decision Support tool for Child and Adolescence Obesity, a system dynamic model developed to explore underlying relationships that contribute to youth obesity.
Outline of behaviour sensitivity analysis
The test presented in this document aims to examine which input assumptions have the greatest impact on modeled policy behavior, unlike previous sensitivity analyses, which focus on numerical relationships between input assumptions and the effectiveness of policy scenarios.
Methods
A sensitivity analysis was conducted, testing 649 input variables between \(\pm\) 20% of an input’s initial value. While each input value may have more extreme values, these input values are considered to be outside of probable values. The model generates 78 BMI prevalence outcomes for each sensitivity analysis, covering 3 BMI categories, 13 age groups, and two genders.
A full depiction of the parameters tested can be found in the S5 Extreme conditions test: model parameters and S6 Extreme conditions test : policy parameters online supplementary materials.
Model behaviour in policy effectiveness is defined as the trend, phase, lag, or amplitude of a policy result over modelled time. Inputs that impact these outcome characteristics are considered to impact the model behaviour.
Outcome Measure
The behavior of each model test was evaluated using the cumulative difference of the first derivative. The following plots explain how this outcome measure is calculated.
Figure panel A shows a plot of three outcomes: the base model, where there are no changes to any initial assumptions; a result that varies an input assumption, producing similar policy behavior; and a second test where varying the input assumption produces different policy behavior. The absolute cumulative difference between the model outcomes is plotted in panel C.
Panel B plots the first derivative of each policy result. The result with similar behavior also has a similar first derivative. Conversely, the result with different behavior has a greater difference in the derivative. The cumulative absolute difference of the first derivative shows a greater difference between the two policy results, highlighting the distinction between behaviors.
While the cumulative absolute difference measure cannot provide insights into the model structure, it is useful to screen input assumptions that impact policy behaviour .
Unlike the extreme test, this analysis focuses on determining which input assumption impacts the policy behaviour.
Example plot for model behaviour
How to interpret the results
The figure below breaks down the summary results shows in the summary plot and the figure above. Larger cumulative difference of the first derivatives reflect model runs that change the model patterned behaviour.
Panel A shows the summary plot for average cumulative difference in the first derivative when energy density of sugar is varied and each of the policy scenarios are implemented in the model. A higher positive value indicates model runs that have a greater impact on model behaviour. This suggests that changes to the assumed energy density for sugar has the greatest impact on the SSB tax policy scenario patterned behaviour.
This conclusion can be further investigated by examining Panel B, the reduction in obesity from policy. In this Panel, the patterned behaviour of the SSB column shows that peaks and troughs of the scenario effectiveness over time deviated from the initial input (dashed lines). This is also reflected in the first derivative plots in Panel C through large difference between sensitivity analysis model results and initial conditions. The cumulative difference between each sampled input and the initial condition first derivative is plotted in Panel D, the average is also plotted as a red line. The average at the end of the model corresponds to the summary plot in Panel A.
In the following result plots, only Panel A is presented; the overarching interpretation to note is
Larger cumulative difference of the first derivatives reflect model runs that change the model patterned behaviour
Results
Early Prevention of Obesity in Children ranking
| Type | Label | Average cumulative difference in derivatives |
|---|---|---|
| Aggregate health effect | Effect Non-core OR | 1.12e-02 |
| Aggregate exposure | Percentage of infants exposed | 9.78e-03 |
| Aggregate health effect | Effect BF OR | 9.28e-03 |
| Fidelity scale up | EPOCH Scale up factor | 8.60e-03 |
| Aggregate health effect | Effect TV OR | 6.78e-03 |
| Relapse | Relapse Rate EPOCH | 3.17e-03 |
| Relapse | Relapse Rate | 2.22e-04 |
| Economic assumptions | Commonwealth Healthcare Costs | 0.00e+00 |
| Economic assumptions | Early Childhood Prevention Cost | 0.00e+00 |
| Economic assumptions | Household Healthcare Costs | 0.00e+00 |
| Economic assumptions | Productivity Loss of guardians from kids missing school | 0.00e+00 |
| Economic assumptions | State Healthcare Costs | 0.00e+00 |
| Economic assumptions | $ per QALY | 0.00e+00 |
| Economic assumptions | Invariant Cost proportions | 0.00e+00 |
| Economic assumptions | Quality of Life Health Utilities per year Reference | 0.00e+00 |
| Note: | ||
| Ranked variables |
Childcare setting intervention variable ranking
| Type | Label | Average cumulative difference in derivatives |
|---|---|---|
| Fidelity scale up | CCI Scale up factor | 2.58e-02 |
| Aggregate exposure | Percentage of childcare centers participating in intervention | 2.58e-02 |
| Aggregate exposure | “Percentage of 3-6 year olds attending formal childcare” | 2.11e-02 |
| Behaviour definition | “Proportion of Fruit Juice in Non-alcoholic beverages” | 1.52e-02 |
| Aggregate health effect | Childcare settings intervention Change in Diet coefficients | 9.97e-03 |
| Relapse | Relapse Rate | 9.94e-03 |
| Behaviour definition | Proportion of Cordial in SSB | 8.78e-03 |
| Behaviour definition | Proportion of Packed snacks in Discretionary foods | 6.25e-03 |
| Behaviour definition | “Proportion of Cakes/muffins/Cookies in Discretionary foods” | 5.16e-03 |
| Aggregate exposure | “Percentage of 2-3 year olds attending formal childcare” | 4.91e-03 |
| Behaviour definition | “Proportion of Chocolate/candy in Discretionary foods” | 3.92e-03 |
| Aggregate health effect | Childcare settings intervention Change in PA coefficients | 1.21e-03 |
| Economic assumptions | “Childcare-based Prevention Cost ($ per child /year)” | 0.00e+00 |
| Economic assumptions | Commonwealth Healthcare Costs | 0.00e+00 |
| Economic assumptions | Household Healthcare Costs | 0.00e+00 |
| Economic assumptions | Productivity Loss of guardians from kids missing school | 0.00e+00 |
| Economic assumptions | State Healthcare Costs | 0.00e+00 |
| Economic assumptions | $ per QALY | 0.00e+00 |
| Economic assumptions | Quality of Life Health Utilities per year Reference | 0.00e+00 |
| Note: | ||
| Ranked variables |
School-based settings interventions ranking
| Type | Label | Average cumulative difference in derivatives |
|---|---|---|
| Fidelity scale up | SI Scale up factor | 3.92e-02 |
| Aggregate exposure | SI Percentage of Schools participating in intervention | 3.92e-02 |
| Behaviour definition | SI percentage of total diet consumed at school | 2.97e-02 |
| Relapse | Relapse Rate | 2.08e-02 |
| Aggregate health effect | SI % change in PA | 9.26e-03 |
| Aggregate exposure | SI Percentage of children attending School | 8.43e-03 |
| Aggregate health effect | SI % Reduction In student Purchases | 2.72e-03 |
| Economic assumptions | Commonwealth Healthcare Costs | 0.00e+00 |
| Economic assumptions | Household Healthcare Costs | 0.00e+00 |
| Economic assumptions | Productivity Loss of guardians from kids missing school | 0.00e+00 |
| Economic assumptions | “School-based Prevention Cost ($ per child /year)” | 0.00e+00 |
| Economic assumptions | State Healthcare Costs | 0.00e+00 |
| Economic assumptions | $ per QALY | 0.00e+00 |
| Economic assumptions | Quality of Life Health Utilities per year Reference | 0.00e+00 |
| Note: | ||
| Ranked variables |
Sports voucher interventions ranking
| Type | Label | Average cumulative difference in derivatives |
|---|---|---|
| Fidelity scale up | SVI Scale up factor | 8.90e-02 |
| Aggregate health effect | Average proportion attributed to vouchers | 8.90e-02 |
| Behaviour definition | Males Average MET Calculated Inputs | 5.89e-02 |
| Behaviour definition | Females Average MET Calculated Inputs | 4.54e-02 |
| Relapse | Relapse Rate | 4.03e-02 |
| Aggregate exposure | “% of population that Registers for vouchers” | 1.85e-02 |
| Behaviour definition | Average Number of Hours of organised sports or activities Weekly | 6.70e-03 |
| Economic assumptions | Commonwealth Healthcare Costs | 0.00e+00 |
| Economic assumptions | Household Healthcare Costs | 0.00e+00 |
| Economic assumptions | Productivity Loss of guardians from kids missing school | 0.00e+00 |
| Economic assumptions | “Sports Vouchers Program cost ($ per child /year)” | 0.00e+00 |
| Economic assumptions | State Healthcare Costs | 0.00e+00 |
| Economic assumptions | $ per QALY | 0.00e+00 |
| Economic assumptions | Quality of Life Health Utilities per year Reference | 0.00e+00 |
| Note: | ||
| Ranked variables |
Sugar-Sweetened Beverge Tax % ranking
| Type | Label | Average cumulative difference in derivatives |
|---|---|---|
| Fidelity scale up | SSB Scale up factor | 1.46e-01 |
| Aggregate health effect | SSB Tax % | 1.46e-01 |
| Relapse | Relapse Rate | 7.60e-02 |
| Behaviour definition | PE Soft drink | 1.74e-02 |
| Behaviour definition | PE Fruit Drink | 9.42e-03 |
| Behaviour definition | PE Cordial | 8.03e-03 |
| Economic assumptions | Commonwealth Healthcare Costs | 0.00e+00 |
| Economic assumptions | Household Healthcare Costs | 0.00e+00 |
| Economic assumptions | Productivity Loss of guardians from kids missing school | 0.00e+00 |
| Economic assumptions | State Healthcare Costs | 0.00e+00 |
| Economic assumptions | $ per QALY | 0.00e+00 |
| Economic assumptions | Quality of Life Health Utilities per year Reference | 0.00e+00 |
| Economic assumptions | Sugar Sweetened Beverage Tax cost 1st year | 0.00e+00 |
| Economic assumptions | Sugar Sweetened Beverage Tax cost subsequent years | 0.00e+00 |
| Note: | ||
| Ranked variables |
Overall Ranking of policy sensitivity
| CCI | EPOCH | SI | SSB | SVI | |
|---|---|---|---|---|---|
| Fidelity scale up | 0.026 | 0.009 | 0.039 | 0.146 | 0.089 |
| Aggregate health effect | 0.007 | 0.009 | 0.003 | 0.146 | 0.089 |
| Relapse | 0.010 | 0.002 | 0.021 | 0.076 | 0.040 |
| Behaviour definition | 0.008 | 0.030 | 0.010 | 0.012 | |
| Aggregate exposure | 0.017 | 0.010 | 0.014 | 0.019 | |
| Economic assumptions | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 |
| Note: | |||||
| Average variable type |
Early Prevention of Obesity in Children
Early Prevention of Obesity in Children Scale up of intervention effects
Early Prevention of Obesity in Children: RCT Intervention effect
Percentage of infants exposed to intervention
Early Prevention of Obesity in Children: Relapse Rate
Childcare setting intervention
Childcare setting intervention Scale up factor
Childcare settings intervention Change in Diet coefficients
Childcare settings intervention Change in physical activity coefficients
Percentage of childcare centers participating in intervention
Definition in food groups
School-based settings interventions
School-based interventions: Relapse Rate
Percentage of Schools participating in intervention
Percentage of total diet consumed at school
School-based intervention Scale up factor
Phyisical activity at school
Reduction In student canteen purchases
Percentage of children attending School
Sports voucher interventions
Sports vouchers Scale up factor
Percentage of the population that Registers for vouchers
Average Number of Hours of organised sports or activities Weekly
Average proportion attributed to vouchers
Average Metabolic Equivalent of Task (MET) for sports Inputs
Sports voucher interventions: Relapse Rate
Sugar-Sweetened Beverge Tax %
Sugar-Sweetened Beverge Scale up factor
Price Elasticity - Cordial
Price Elasticity - Fruit Drink
Price Elasticity - Soft drink
Sugar-Sweetened Beverge Tax: Relapse Rate
Sugar-Sweetened Beverge Tax %
## R version 4.3.2 (2023-10-31)
## Platform: x86_64-pc-linux-gnu (64-bit)
## Running under: Pop!_OS 20.04 LTS
##
## Matrix products: default
## BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0
## LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0
##
## locale:
## [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
## [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
## [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
## [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
## [9] LC_ADDRESS=C LC_TELEPHONE=C
## [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
##
## time zone: Australia/Sydney
## tzcode source: system (glibc)
##
## attached base packages:
## [1] stats graphics grDevices utils datasets methods base
##
## other attached packages:
## [1] kableExtra_1.3.4 scales_1.2.1 ggpubr_0.6.0 ggh4x_0.2.5
## [5] patchwork_1.1.3 openxlsx_4.2.5.2 lubridate_1.9.2 forcats_1.0.0
## [9] stringr_1.5.0 dplyr_1.1.2 purrr_1.0.2 readr_2.1.4
## [13] tidyr_1.3.0 tibble_3.2.1 ggplot2_3.4.3 tidyverse_2.0.0
## [17] progress_1.2.2
##
## loaded via a namespace (and not attached):
## [1] gtable_0.3.4 xfun_0.40 bslib_0.5.1 rstatix_0.7.2
## [5] tzdb_0.4.0 vctrs_0.6.3 tools_4.3.2 generics_0.1.3
## [9] fansi_1.0.4 highr_0.10 pkgconfig_2.0.3 webshot_0.5.5
## [13] lifecycle_1.0.3 farver_2.1.1 compiler_4.3.2 munsell_0.5.0
## [17] carData_3.0-5 htmltools_0.5.6 sass_0.4.7 yaml_2.3.7
## [21] pillar_1.9.0 car_3.1-2 crayon_1.5.2 jquerylib_0.1.4
## [25] cachem_1.0.8 abind_1.4-5 tidyselect_1.2.0 rvest_1.0.3
## [29] zip_2.3.0 digest_0.6.33 stringi_1.7.12 bookdown_0.35.1
## [33] labeling_0.4.2 rmdformats_1.0.4 fastmap_1.1.1 grid_4.3.2
## [37] colorspace_2.1-0 cli_3.6.1 magrittr_2.0.3 utf8_1.2.3
## [41] broom_1.0.5 withr_2.5.0 prettyunits_1.1.1 backports_1.4.1
## [45] timechange_0.2.0 rmarkdown_2.24 httr_1.4.7 ggsignif_0.6.4
## [49] hms_1.1.3 evaluate_0.21 knitr_1.44 viridisLite_0.4.2
## [53] rlang_1.1.1 Rcpp_1.0.11 glue_1.6.2 xml2_1.3.5
## [57] svglite_2.1.1 rstudioapi_0.15.0 jsonlite_1.8.7 R6_2.5.1
## [61] systemfonts_1.0.4